Non-Volatile Memory Express (NVMe) is a leading technology for directly connecting storage (e.g., a solid state drive (SSD) or Flash drive) to a host (e.g., a computer or server) via a Peripheral Component Interconnect Express (PCIe) interface. The directly-connected host transfers data to and from an NVMe storage device using direct memory access (DMA).
When implemented in a network, for example, the host (e.g., a server) directly attached to the NVMe storage device may also be connected to other computers (e.g., clients) through the network. The client computers, which are not directly attached to the NVMe storage device, access the NVMe storage device in a conventional manner through the network and the server.
RDMA (Remote Direct Memory Access) and NVMe-over-RDMA are known protocols for enabling remotely-connected hosts to transfer data to and from the NVMe storage device. For example, in a write operation, the RDMA protocol communicates data from the system memory of the client over a network to the system memory of the server. At the server, the data is buffered in the system memory of the server, and then the NVMe storage device may access the buffered data. If the server is attached to many clients and each client wishes to access the storage device(s) of the server, the system memory of the server may become a bottleneck.
A known improvement of the RDMA method partially lessens the bottleneck caused by the system memory of the server by utilizing a buffer memory associated with the controller of the NVMe storage device. However, this known method still depends on the involvement of the system memory of the server host and further requires use of special, proprietary, or non-standard NVMe storage devices, limiting the compatibility of this known method in various system implementations.
Therefore, improvements to networked-NVMe transfer approaches are desirable.